Analysis and Development of Augmented Reality Applications for the Dissemination of Cultural Heritage

Tesis por compendio[ES] La RA consiste en la superposición de elementos virtuales sobre el entorno real, de manera que el usuario percibe estos elementos como si formaran parte de la realidad que está visualizando. Las aplicaciones de RA en dispositivos móviles permiten visualizar el contenido virtual a través de la cámara del dispositivo. La RA es una herramienta de divulgación muy potente ya que permite añadir a la realidad cualquier tipo de información, desde un simple texto informativo a un modelo 3D interactivo. Tiene infinitas utilidades, puede servir de guía en un museo, puede mostrar la recreación de un monumento destruido, o como en el caso de estudio aquí presentado, ayudar a la interpretación de pinturas rupestres. Esta tesis parte de la idea de que la RA puede mejorar mucho la interpretación del arte rupestre sin alterar ni dañar las pinturas. Puede servir para atraer a un público mayor, dar a conocer la historia de las pinturas rupestres y que al mismo tiempo el visitante tenga una experiencia mucho más enriquecedora. A lo largo de la tesis se ha estudiado en profundidad la técnica de visualización de RA mediante dispositivos móviles. Se han analizado las diferentes librerías de programación mediante casos de estudio en entornos reales y examinado los factores que pueden afectar al reconocimiento de las pinturas. Se ha desarrollado una aplicación de RA aplicada a un caso real de pinturas rupestres y posteriormente ha sido evaluada por un grupo de personas. Finalmente, se ha estudiado el efecto de la luz solar y sus cambios a lo largo del día sobre el reconocimiento de imágenes en entornos al aire libre. Este trabajo proporciona un punto de partida para el desarrollo de aplicaciones de RA aplicadas a la difusión del patrimonio cultural, especialmente centrado en el arte rupestre, un entorno que sufre de unas dificultades añadidas debido a su localización, dificultad de reconocimiento de puntos característicos en las pinturas y los cambios en la luz solar, problemas que se han tratado de resolver a lo largo del estudio. Las principales conclusiones han sido muy favorables, partiendo de librerías de programación disponibles y gratuitas. Se han podido desarrollar un conjunto de aplicaciones de RA en diferentes lugares. Las valoraciones han sido muy positivas, los usuarios que han probado las aplicaciones afirman que la interpretación de las pinturas les resulta más fácil y consiguen entender mejor el propósito de las mismas. El principal inconveniente encontrado es la falta de conocimiento sobre esta técnica y la pérdida de realismo en algunos casos debido a la oclusión, es decir, que los objetos virtuales no se posicionen por detrás de los objetos reales. La buena noticia es que esta tecnología evoluciona muy rápido y durante el desarrollo de la tesis ha habido avances muy grandes, entre ellos, el desarrollo de nuevas librerías de programación desarrolladas por Google y Apple, que proporcionan las herramientas necesarias para crear aplicaciones muy potentes e immersivas, donde el usuario se sentirá parte de los entornos creados.[CA] La RA consisteix en la superposició d'elements virtuals sobre l'entorn real, de manera que l'usuari percep aquests elements com si formaren part de la realitat que està visualitzant. Les aplicacions de RA en dispositius mòbils permeten visualitzar el contingut virtual a través de la cambra del dispositiu. La RA és una eina de divulgació molt potent ja que permet afegir a la realitat qualsevol tipus d'informació, des d'un simple text informatiu a un model 3D interactiu. Té infinites utilitats, pot servir de guia en un museu, pot mostrar la recreació d'un monument destruït, o com en el cas d'estudi ací presentat, ajudar a la interpretació de pintures rupestres. Aquesta tesi parteix de la idea que la RA pot millorar molt la interpretació de l'art rupestre sense alterar ni danyar les pintures. Pot servir per a atraure a un públic major, donar a conéixer la història de les pintures rupestres i que al mateix temps el visitant tinga una experiència molt més enriquidora. Al llarg de la tesi s'ha estudiat en profunditat la tècnica de visualització de RA mitjançant dispositius mòbils. S'han analitzat les diferents llibreries de programació mitjançant casos d'estudi en entorns reals i analitzat els factors que poden afectar el reconeixement de les pintures. S'ha desenvolupat una aplicació de RA aplicada a un cas real de pintures rupestres i posteriorment ha sigut avaluada per un grup de persones. Finalment, s'ha estudiat l'efecte de la llum solar i els seus canvis al llarg del dia sobre el reconeixement d'imatges en entorns a l'aire lliure. Aquest treball proporciona un punt de partida per al desenvolupament d'aplicacions de RA aplicades a la difusió del patrimoni cultural, especialment centrat en l'art rupestre, un entorn que pateix d'unes dificultats afegides a causa de la seua localització, dificultat de reconeixement de punts característics en les pintures i els canvis en la llum solar, problemes que s'han tractat de resoldre al llarg de l'estudi. Les principals conclusions han sigut molt favorables, partint de llibreries de programació disponibles i gratuïtes. S'han pogut desenvolupar un conjunt d'aplicacions de RA en diferents llocs. Les valoracions han sigut molt positives, els usuaris que han provat les aplicacions afirmen que la interpretació de les pintures els resulta més fàcil i aconsegueixen entendre millor el propòsit d'aquestes. El principal inconvenient trobat és la falta de coneixement sobre aquesta tècnica i la perduda de realisme en alguns casos a causa de l'oclusió, és a dir, que els objectes virtuals no es posicionen per darrere dels objectes reals. La bona notícia és que aquesta tecnologia evoluciona molt ràpid i durant el desenvolupament de la tesi hi ha hagut avanços molt grans, entre ells, el desenvolupament de noves llibreries de programació per Google i Apple, que proporcionen les eines necessàries per a crear aplicacions molt potents i immersives, on l'usuari se sentirà part dels entorns creats.[EN] AR consists of superimposing virtual elements on the real environment, so that the user perceives these elements as if they were part of the reality they are looking at. AR applications on smartphones allow virtual content to be visualised through the device's camera. AR is a very powerful tool for dissemination as it allows any type of information to be added to reality, from a simple informative text to an interactive 3D model. It can be used as a guide in a museum, it can show the recreation of a destroyed monument, or, as in the case study presented here, it can help in the interpretation of cave paintings. This thesis is based on the idea that AR can greatly enhance the interpretation of rock art without affecting or damaging the paintings. It can be used to attract a wider audience, to introduce the history of the rock art paintings and at the same time provide the visitor with a much more enriching experience. Throughout the thesis, the technique of AR visualisation using mobile devices has been studied in-depth. The different programming libraries have been analysed by means of case studies in real environments as well as the factors that can affect the paintings recognition. An AR application applied to a real case of rock art paintings has been developed and subsequently evaluated by a group of people. Finally, the effect of sunlight and its changes throughout the day on image recognition in outdoor environments has been studied. This work provides a starting point for the AR applications development applied to the dissemination of cultural heritage, especially focused on rock art, an environment that suffers from additional difficulties due to its location, the difficulty of characteristic points recognition and changes in sunlight, problems that have been tried to solve throughout the study. The main outcomes have been very favourable, using freely available programming libraries, and it has been possible to develop a set of AR applications in different places. The evaluations have been very positive, with users who have tested the applications confirming that the interpretation of the paintings is easier for them and they can better understand the purpose of the paintings. The major drawback is the lack of knowledge about this technique and the loss of realism in some cases due to occlusion, i.e. the virtual objects are not positioned behind the real objects. The good news is that this technology is evolving very fast and during the development of the thesis there have been great advances, among them, the development of new programming libraries developed by Google and Apple, which provide the necessary tools to create very powerful and immersive applications, where the user will feel part of the virtual environments created.Blanco Pons, S. (2021). Analysis and Development of Augmented Reality Applications for the Dissemination of Cultural Heritage [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/178895TESISCompendi

Augmented reality application assessment for disseminating rock art

[EN] Currently, marker-based tracking is the most used method to develop augmented reality (AR) applications (apps). However, this method cannot be applied in some complex and outdoor settings such as prehistoric rock art sites owing to the fact that the usage of markers is restricted on site. Thus, natural feature tracking methods have to be used. There is a wide range of libraries to develop AR apps based on natural feature tracking. In this paper, a comparative study of Vuforia and ARToolKit libraries is carried out, analysing factors such as distance, occlusion and lighting conditions that affect user experience in both indoor and outdoor environments, and eventually the app developer. Our analysis confirms that Vuforia¿s user experience indoor is better, faster and flicker-free whether the images are properly enhanced, but it does not work properly on site. Therefore, the development of AR apps for complex outdoor environments such as rock art sites should be performed with ARToolKit.The authors gratefully acknowledge the support from the Spanish Ministerio de Economia y Competitividad to the project HAR2014-59873-R. Similarly, the authors want to express their gratitude to the General Directorate of Culture and Heritage, Conselleria d'Educacio, Investigacio, Cultura i Esport, Generalitat Valenciana for letting us access and carry out research at the archaeological site.Blanco-Pons, S.; Carrión-Ruiz, B.; Lerma, JL. (2018). Augmented reality application assessment for disseminating rock art. Multimedia Tools and Applications. 78(8):10265-10286. https://doi.org/10.1007/s11042-018-6609-xS1026510286788Alahi A., Ortiz R., Vandergheynst P (2012) FREAK: fast retina keypoint. Comput Vis Pattern Recognit 510–517 . doi: https://doi.org/10.1109/CVPR.2012.6247715Amin D, Govilkar S (2015) Comparative study of augmented reality Sdk’S. Int J Comput Sci Appl 5:11–26. https://doi.org/10.1227/01.NEU.0000297044.82035.57ARCore ARCore - Google Developer | ARCore | Google Developers. https://developers.google.com/ar/ . Accessed 26 Jun 2018ARKit ARKit - Apple Developer. https://developer.apple.com/arkit/ . Accessed 26 Jun 2018ARToolkit (2017) ARToolkit. https://archive.artoolkit.org/ . Accessed 2 Oct 2017ARToolkit (2017) About. https://artoolkit.org/about-artoolkit . Accessed 11 Apr 2017ARToolkit (2017) Documentation. https://artoolkit.org/documentation/ . Accessed 12 Apr 2017ArUco ArUco: A minimal library for Augmented Reality applications based on OpenCV | Aplicaciones de la Visión Artificial. https://www.uco.es/investiga/grupos/ava/node/26 . Accessed 19 Apr 2018Azuma R (1997) A survey of augmented reality. Presence Teleoperators Virt Environ 6:355–385 . doi: 10.1.1.30.4999Azuma R, Baillot Y, Feiner S et al (2001) Recent advances in augmented reality. Ieee Comput Graph Appl 34–47. doi: https://doi.org/10.4061/2011/908468Blanco-Novoa O, Fernandez-Carames TM, Fraga-Lamas P, Vilar-Montesinos M (2018) A practical evaluation of commercial industrial augmented reality systems in an industry 4.0 shipyard. IEEE Access 6:1–1. https://doi.org/10.1109/ACCESS.2018.2802699Blanco-Pons S, Carrión-Ruiz B, Lerma JL (2016) Review of augmented reality and virtual reality techniques in rock art. Proc 8th Int Congress Archaeol Comput Graph Cult Herit Innov ‘ARQUEOLÓGICA 2.0L: 176–183Brancati N, Caggianese G, Frucci M et al (2017) Experiencing touchless interaction with augmented content on wearable head-mounted displays in cultural heritage applications. Pers Ubiquitous Comput 21:203–217. https://doi.org/10.1007/s00779-016-0987-8Cagalaban G, Kim S (2010) Multiple object tracking in unprepared environments using combined feature for augmented reality applications. Springer, Berlin, HeidelbergCamera-Calibration Camera Calibration App for Android [ARToolkit]. https://archive.artoolkit.org/documentation/doku.php?id=4_Android:android_camera_calibration . Accessed 16 Oct 2017Carmigniani J, Furht B, Anisetti M et al (2011) Augmented reality technologies, systems and applications. Multimed Tools Appl 51:341–377. https://doi.org/10.1007/s11042-010-0660-6Carrión-Ruiz B, Blanco-Pons S, Lerma JL (2016) Digital image analysis of the visible region through simulation of rock art paintings. Proc 8th Int Congress Archaeol Comput Graph, Cult Heritage Innov ‘ARQUEOLÓGICA 2.0.’: 169–175Chen CY, Chang BR, Sen HP (2014) Multimedia augmented reality information system for museum guidance. Pers Ubiquitous Comput 18:315–322. https://doi.org/10.1007/s00779-013-0647-1CRYENGINE CRYENGINE | The complete solution for next generation game development by Crytek. https://www.cryengine.com/ . Accessed 7 Jun 2017Domingo I, Carrión B, Blanco S, Lerma JL (2015) Evaluating conventional and advanced visible image enhancement solutions to produce digital tracings at el Carche rock art shelter. Digit Appl Archaeol Cult Herit 2:79–88. https://doi.org/10.1016/j.daach.2015.01.001Dos Santos AB, Dourado JB, Bezerra A (2016) ARToolkit and Qualcomm Vuforia: An Analytical Collation. Proc - 18th Symp Virt Augment Real SVR 2016:229–233. https://doi.org/10.1109/SVR.2016.46DroidAR (2017) DroidAR by bitstars. https://bitstars.github.io/droidar/ . Accessed 10 Dec 2017Engine U (2017) Unreal Engine. https://www.unrealengine.com/ . Accessed 10 Oct 2017Fiala M (2005) ARTag, a fiducial marker system using digital techniques. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn 2:590–596. https://doi.org/10.1109/CVPR.2005.74Fischer J, Eichler M, Bartz D, Straßer W (2007) A hybrid tracking method for surgical augmented reality. Comput Graph 31:39–52. https://doi.org/10.1016/j.cag.2006.09.007González C, Vallejo D, Albusac J, Castro J (2011) Realidad Aumentada. Un enfoque práctico con ARToolKit y Blender. 2–120Gutierrez JM, Molinero MA, Soto-Martín O, Medina CR (2015) Augmented reality technology spreads information about historical graffiti in temple of Debod. Procedia Comput Sci 75:390–397. https://doi.org/10.1016/j.procs.2015.12.262Haladová ZB, Szemzö R, Kovačovský T, Žižka J (2015) Utilizing Multispectral Scanning and Augmented Reality for Enhancement and Visualization of the Wooden Sculpture Restoration Process. Procedia Comput Sci 67:340–347. https://doi.org/10.1016/j.procs.2015.09.278Jamali SS, Shiratuddin MF, Wong KW, Oskam CL (2015) Utilising mobile-augmented reality for learning human anatomy. Procedia - Soc Behav Sci 197:659–668. https://doi.org/10.1016/j.sbspro.2015.07.054Khan D, Ullah S, Rabbi I (2015) Factors affecting the design and tracking of ARToolKit markers. Comput Stand Interf 41:56–66. https://doi.org/10.1016/j.csi.2015.02.006Khan D, Ullah S, Yan D et al (2018) Robust tracking through the design of high quality fiducial markers: an optimization tool for ARToolKit. IEEE Access 4:22421–22433. https://doi.org/10.1109/ACCESS.2018.2801028Kim SL, Suk HJ, Kang JH, et al (2014) Using unity 3D to facilitate mobile augmented reality game development. Internet things (WF-IoT), 2014 IEEE World Forum 21–26 . doi: https://doi.org/10.1109/WF-IoT.2014.6803110Kounavis CD, Kasimati AE, Zamani ED (2012) Enhancing the tourism experience through mobile augmented reality: challenges and prospects. Int J Eng Bus Manag 4:1–6. https://doi.org/10.5772/51644La Delfa GC, Monteleone S, Catania V et al (2016) Performance analysis of visualmarkers for indoor navigation systems. Front Inf Technol Electron Eng 17:730–740. https://doi.org/10.1631/FITEE.1500324Liu S, Ge S, Yu H (2016) Research on Robustness recognition algorithms in augmented reality. 3rd Int Conf Inf Sci Control Eng: 547–552. doi: https://doi.org/10.1109/ICISCE.2016.123Lowe DG (2004) Distinctive image features from scale invariant keypoints. Int J Comput Vis 60:91–11020042. https://doi.org/10.1023/B:VISI.0000029664.99615.94Lytridis C, Tsinakos A, Kazanidis I (2018) ARTutor—an augmented reality platform for interactive distance learning. Educ Sci 8:6. https://doi.org/10.3390/educsci8010006Marchand E, Uchiyama H, Spindler F et al (2016) Pose estimation for augmented reality : a hands-on survey. IEEE Trans Vis Comput Graph 22:2633–2651. https://doi.org/10.1109/TVCG.2015.2513408Martínez R, Villaverde V (2002) La cova dels cavalls en el Barranc de la ValltortaMarto AGR, Sousa AA, de Gonçalves A (2017) DinofelisAR demo augmented reality based on natural features. 12a Conferência Ibérica Sist e Tecnol Informação, Lisboa 64:852–861. https://doi.org/10.1016/j.procs.2015.08.638Moreels P, Perona P (2007) Evaluation of feature detectors and descriptors based on 3D objects. Int J Comput Vis 73:263–284. https://doi.org/10.1007/s11263-006-9967-1Pierdicca R, Frontoni E, Zingaretti P et al (2015) Making visible the invisible. augmented reality visualization for 3D reconstructions of archaeological sites. Augment Virt Real Sec Int Conf AVR 2015 9254:25–37. https://doi.org/10.1007/978-3-319-22888-4Rabbi I, Ullah S, Javed M, Zen K (2014) Analysis of ARToolKit fiducial markers attributes for robust tracking. 1st Int Conf Recent Trends Inf Commun Technol Anal 281–290Radkowski R, Oliver J (2013) Natural feature tracking augmented reality for on-site assembly assistance systems. In: Shumaker R (ed) Virtual, Augmented and Mixed Reality. Systems and Applications. VAMR 2013. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg, pp 281–290Ridel B, Reuter P, Laviole J et al (2014) The revealing flashlight: interactive spatial augmented reality for detail exploration of cultural heritage artifacts. J Comput Cult Herit 7(6):1–6:18. https://doi.org/10.1145/2611376Seo J, Shim J, Choi JH, et al (2011) Enhancing marker-based AR technology. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 6773 LNCS:97–104 . doi: https://doi.org/10.1007/978-3-642-22021-0_12Seo J, Shim J, Choi JH et al (2011) Enhancing marker-based AR technology. In: International conference on virtual and mixed reality. virtual and mixed reality - new trends. Springer, Berlin, Heidelberg, pp 97–104Siltanen S (2015) Diminished reality for augmented reality interior design. Vis Comput 33:1–16. https://doi.org/10.1007/s00371-015-1174-zSörös G, Seichter H, Rautek P, Gröller E (2011) Augmented visualization with natural feature tracking. Proc 10th Int Conf Mob Ubiquitous Multimed 4–12. doi: https://doi.org/10.1145/2107596.2107597Uchiyama H, Marchand E (2012) Object detection and pose tracking for augmented reality: recent approaches. 18th Korea-Japan Jt Work Front Comput Vis 1–8Unity Unity. https://unity3d.com/es . Accessed 12 Oct 2017Vuforia (2017) Vuforia. https://www.vuforia.com/ . Accessed 2 Oct 2017Vuforia (2017) Vuforia-VuMark. https://library.vuforia.com/articles/Training/VuMark . Accessed 4 Apr 2017Vuforia (2017) Image targets. https://library.vuforia.com/articles/Training/Image-Target-Guide . Accessed 11 Apr 2017Wang H, Qin J, Zhang F (2015) A new interaction method for augmented reality based on ARToolKit. 2015 8th Int Congr Image Signal Process 578–583. doi: https://doi.org/10.1109/CISP.2015.7407945Wang G, Wang B, Zhong F et al (2015) Global optimal searching for textureless 3D object tracking. Vis Comput 31:979–988. https://doi.org/10.1007/s00371-015-1098-7Wu S, Oerlemans A, Bakker EM, Lew MS (2017) A comprehensive evaluation of local detectors and descriptors. Signal Process Image Commun 59:150–167. https://doi.org/10.1016/J.IMAGE.2017.06.010Xu Y, Wu Y, Zhou H, View M (2018) Multi-scale Voxel Hashing and Efficient 3D Representation for Mobile Augmented Reality. Cvpr 1618–1625 . doi: https://doi.org/10.1109/CVPRW.2018.0020

REVISIÓN DE LAS TÉCNICAS DE REALIDAD AUMENTADA Y REALIDAD VIRTUAL EN EL ARTE RUPESTRE

[EN] The usage of augmented reality (AR) and virtual reality (VR) technologies began to grow when smartphones appeared. Until then, the number of portable devices capable of incorporating these technologies was reduced. Video games are the main field where these technologies are applied, but in other fields such as in archaeology, these technologies can offer many advantages. Ruins reconstruction, ancient life simulation, highly detailed 3D models visualisation of valuable objects from the past or even user free movement in missing places are just some examples found in literature. This paper reviews the latest visualisation technologies and their applicability to the rock art field. The main purpose is to disseminate rock art paintings through AR and VR applications. After the image-based three-dimensional (3D) modelling is obtained, an interactive visit to a shelter for displaying rock art paintings is presented. This is one of examples developed in this paper that pretends to apply the revised AR and VR techniques. In addition, an example of AR is developed that can be easily adapted to further applications displaying rock art paintings.[ES] El uso de las tecnologías de realidad virtual (RV) y realidad aumentada (RA) comenzó a crecer con la aparición de los teléfonos inteligentes. Hasta entonces, había pocos dispositivos portátiles capaces de incorporar estas tecnologías. Principalmente estas tecnologías se aplican en la creación de videojuegos, aunque en otros campos como la arqueología, estas tecnologías tienen mucho que ofrecer. Con estas tecnologías es posible mostrar la reconstrucción de ruinas, la simulación de otra época, modelos 3D altamente detallados de objetos de valor del pasado o incluso trasladar al usuario a lugares que ya no existen. Este trabajo presenta una revisión de las últimas tecnologías de visualización y su aplicabilidad en el campo del arte rupestre, con el objetivo de divulgar las pinturas rupestres a través de aplicaciones de RA y RV. Después del modelado 3D basado en imágenes, se presenta una visita interactiva a un refugio para la visualización de las pinturas rupestres. Este es uno de los ejemplos desarrollados en este trabajo que pretende aplicar las técnicas revisadas de RA y RV. Además, se detalla un sencillo ejemplo de RA que puede fácilmente adaptarse a otras aplicaciones que muestren pinturas rupestres.Blanco Pons, S.; Carrión Ruiz, B.; Lerma, JL. (2016). REVIEW OF AUGMENTED REALITY AND VIRTUAL REALITY TECHNIQUES IN ROCK ART. En 8th International congress on archaeology, computer graphics, cultural heritage and innovation. Editorial Universitat Politècnica de València. 176-183. https://doi.org/10.4995/arqueologica8.2016.3561OCS17618

ÁNALISIS DE IMAGEN DIGITAL DE LA REGIÓN VISIBLE MEDIANTE SIMULACIÓN DE PINTURAS DE ARTE RUPESTRE

[EN] Non-destructive rock art recording techniques are getting special attention in the last years, opening new research lines in order to improve the level of documentation and understanding of our rich legacy. This paper applies the principal component analysis (PCA) technique in images that include wavelengths between 400-700 nm (visible range). Our approach is focused on determining the difference provided by the image processing of the visible region through four spectral images versus an image that encompasses the entire visible spectrum. The images were taken by means of optical filters that take specific wavelengths and exclude parts of the spectrum. Simulation of rock art is prepared in laboratory. For this purpose, three different pigments were made simulating the material composition of rock art paintings. The advantages of studying the visible spectrum in separate images are analysed. In addition, PCA is applied to each of the images to reduce redundant data. Finally, PCA is applied to the image that contains the entire visible spectrum and is compared with previous results. Through the results of the four visible spectral images one can begin to draw conclusions about constituent painting materials without using decorrelation techniques.[ES] Las técnicas documentación no destructivas de arte rupestre están recibiendo especial atención en los últimos años, abriendo nuevas líneas de investigación para la mejora del nivel de documentación y comprensión de nuestro patrimonio. Este artículo aplica la técnica de análisis de componentes principales (ACP) en imágenes de longitudes de onda entre 400-700 nm (rango visible). Nuestro enfoque se centra en determinar las diferencias proporcionadas por el procesamiento de imágenes de la región del visible a través de cuatro imágenes espectrales y una imagen que abarca el espectro visible completo. Las imágenes se tomaron con filtros ópticos que recogen longitudes de onda determinadas y excluyen parte del espectro.Se preparó una simulación de pigmentos de arte ruprestre en laboratorio. Para ello se hicieron tres tipos de pigmentos diferentes simulando la composición del material de las pinturas rupestres. Se analizaron las ventajas de estudiar el espectro visible en imágenes separadas. Además, se aplicó ACP a cada una de las imágenes para reducir la información redundante. Finalmente, se realizó ACP a la imagen que abarca todo el espectro visible y se comparó con los resultados anteriores. A través de los resultados de las cuatro imágenes espectrales del visible se pueden sacar conclusiones sobre los materiales que constituyen la pintura sin aplicar técnicas de decorrelación.Carrión-Ruiz, B.; Blanco-Pons, S.; Lerma García, JL. (2016). DIGITAL IMAGE ANALYSIS OF THE VISIBLE REGION THROUGH SIMULATION OF ROCK ART PAINTINGS. En 8th International congress on archaeology, computer graphics, cultural heritage and innovation. Editorial Universitat Politècnica de València. 169-175. https://doi.org/10.4995/arqueologica8.2016.3560OCS16917

Design and implementation of an augmented reality application for rock art visualization in Cova dels Cavalls (Spain)

[EN] Prehistoric rock art paintings, specifically rock-shelters exposed to environmental and anthropogenic factors, are usually faint and severely damaged, being them difficult to identify and understand by visitors. Augmented Reality (AR) supplements reality with virtual information superimposed onto the real world. This sensor-based technology in smartphones/tablets can improve the paintings experience displaying the 2D digital tracings overlapped onto the real scene (rock with faint paintings). This paper presents an AR application (app) developed in Cova dels Cavalls that shows a recreation of a possible original composition full of motifs with descriptive information to improve current guided tour user experiences. This case study aims to evaluate the rock art AR app targeting non-expert visitors as a means of improving rock art knowledge and sensibility of a fragile archaeological UNESCO Work Heritage site. To achieve this, a variety of participants with different backgrounds and interests tested the AR app on site and answered a complete questionnaire about the use of AR mobile apps. Overall, the results showed great acceptance of this AR app, mainly because in addition to adding new information interactively, it helps to identify the rock art motifs, as well as to recognise them quickly, improving their understanding.The authors gratefully acknowledge the support from the Spanish Ministerio de Economia y Competitividad to the project HAR201459873-R. The authors acknowledge the authorisation of the Conselleria d'Educacio, Investigacio, Cultura i Esports the chance to carry out research at this exceptional archaeological site.Blanco-Pons, S.; Carrión-Ruiz, B.; Lerma, JL.; Villaverde, V. (2019). Design and implementation of an augmented reality application for rock art visualization in Cova dels Cavalls (Spain). Journal of Cultural Heritage. 39:177-185. https://doi.org/10.1016/j.culher.2019.03.014S1771853

Rock art dissemination through augmented reality mobile applications: a practical approach

[EN] Rock art of the Mediterranean Basin of the Iberian Peninsula was declared World Heritage Site by UNESCO, it has an outstanding cultural value hence its dissemination is essential. Over the years, rock art paintings have been deteriorated and even some have disappeared. Consequently, it is quite difficult to recognise them in situ without experts. At this point, non-invasive and novel techniques can offer great potential, on the one hand, avoiding direct contact with the surface promoting its conservation, and on the other hand, through dissemination and analysis of these elements. One of the most promising techniques is Augmented Reality (AR). By means of AR applications, the visitor can see any kind of virtual content related to cultural heritage, through its mobile or tablet screen. Thus, any virtual content that is considered of interest can be added, offering in this way additional information that complements the real element. In this study, two approaches are presented for the rock art dissemination of the Cova dels Cavalls. The first is based on a marker recognition, which launches the application that displays an image of the motif and a descriptive text about it. In the second proposal, the camera device recognises the real image of the painting and the virtual information is overlapped. The study aims at evaluating the use of these AR techniques applied to rock art dissemination and shows the methodology for its development[ES] El Arte rupestre del arco mediterráneo de la Península Ibérica declarado Patrimonio Mundial por la UNESCO, tiene un gran valor cultural por lo que su difusión resulta esencial. Con el paso de los años, las pinturas se han ido deteriorando e incluso algunas han desaparecido. Es por ello que resulta muy difícil reconocerlas en los yacimientos sin la ayuda de expertos. Es en este punto en el que las técnicas no invasivas y más novedosas pueden ofrecer un gran potencial, no solo a la hora de evitar el contacto directo con la superficie fomentando así su conservación, sinó también por la difusión y análisis de estos elementos. Una de las técnicas más prometedoras en este sentido es la Realidad Aumentada (RA) ya que, gracias a ella, el visitante puede ver cualquier tipo de contenido virtual relacionado con el patrimonio, a través de la pantalla de su móvil o tableta. Así, todo contenido virtual que se considere de interés puede ser añadido, ofreciendo de esta manera información adicional que complementa al elemento real. En este estudio se presentan dos propuestas de RA planteadas para la difusión de las pinturas rupestres de la Cova dels Cavalls. La primera está basada en el reconocimiento de un marcador, el cual activa la aplicación mostrando al visitante la imagen 2D de uno de los motivos y un texto descriptivo del mismo. En la segunda propuesta, la cámara del dispositivo reconoce la imagen real de la pintura rupestre y superpone la información virtual. Con este estudio se pretende valorar el uso de estas técnicas de RA aplicadas a la difusión de arte rupestre y mostrar la metodología para su desarrollo.Los autores agradecen el apoyo del Ministerio de Economía y Competitividad al proyecto HAR2014- 59873-R.Blanco Pons, S.; Lerma García, JL. (2017). Difusión del Arte rupestre a través de aplicaciones móviles de realidad aumentada: un enfoque práctico. En Primer Congreso en Ingeniería Geomática. Libro de actas. Editorial Universitat Politècnica de València. 1-7. https://doi.org/10.4995/CIGeo2017.2017.6598OCS1

Augmented Reality Markerless Multi-Image Outdoor Tracking System for the Historical Buildings on Parliament Hill

[EN] Augmented Reality (AR) applications have experienced extraordinary growth recently, evolving into a well-established method for the dissemination and communication of content related to cultural heritage¿including education. AR applications have been used in museums and gallery exhibitions and virtual reconstructions of historic interiors. However, the circumstances of an outdoor environment can be problematic. This paper presents a methodology to develop immersive AR applications based on the recognition of outdoor buildings. To demonstrate this methodology, a case study focused on the Parliament Buildings National Historic Site in Ottawa, Canada has been conducted. The site is currently undergoing a multiyear rehabilitation program that will make access to parts of this national monument inaccessible to the public. AR experiences, including simulated photo merging of historic and present content, are proposed as one tool that can enrich the Parliament Hill visit during the rehabilitation. Outdoor AR experiences are limited by factors, such as variable lighting (and shadows) conditions, caused by changes in the environment (objects height and orientation, obstructions, occlusions), the weather, and the time of day. This paper proposes a workflow to solve some of these issues from a multi-image tracking approach.This work has been developed under the framework of the New Paradigms/New Tools for Heritage Conservation in Canada, a project funded through the Social Sciences and Humanities Research Council of Canada (SSHRC).Blanco-Pons, S.; Carrión-Ruiz, B.; Duong, M.; Chartrand, J.; Fai, S.; Lerma, JL. (2019). Augmented Reality Markerless Multi-Image Outdoor Tracking System for the Historical Buildings on Parliament Hill. Sustainability. 11(16):1-15. https://doi.org/10.3390/su11164268S1151116Bekele, M. K., Pierdicca, R., Frontoni, E., Malinverni, E. S., & Gain, J. (2018). A Survey of Augmented, Virtual, and Mixed Reality for Cultural Heritage. Journal on Computing and Cultural Heritage, 11(2), 1-36. doi:10.1145/3145534Gimeno, J., Portalés, C., Coma, I., Fernández, M., & Martínez, B. (2017). Combining traditional and indirect augmented reality for indoor crowded environments. A case study on the Casa Batlló museum. Computers & Graphics, 69, 92-103. doi:10.1016/j.cag.2017.09.001Kolivand, H., El Rhalibi, A., Shahrizal Sunar, M., & Saba, T. (2018). ReVitAge: Realistic virtual heritage taking shadows and sky illumination into account. Journal of Cultural Heritage, 32, 166-175. doi:10.1016/j.culher.2018.01.020Amakawa, J., & Westin, J. (2017). New Philadelphia: using augmented reality to interpret slavery and reconstruction era historical sites. International Journal of Heritage Studies, 24(3), 315-331. doi:10.1080/13527258.2017.1378909Kim, J.-B., & Park, C. (2011). Development of Mobile AR Tour Application for the National Palace Museum of Korea. Lecture Notes in Computer Science, 55-60. doi:10.1007/978-3-642-22021-0_7Barrile, V., Fotia, A., Bilotta, G., & De Carlo, D. (2019). Integration of geomatics methodologies and creation of a cultural heritage app using augmented reality. Virtual Archaeology Review, 10(20), 40. doi:10.4995/var.2019.10361Analysis of Tracking Accuracy for Single-Camera Square-Marker-Based Tracking. In Third Workshop on Virtual and Augmented Reality of the GI-Fachgruppe VR/AR, Koblenz, Germany, 2006http://campar.in.tum.de/Chair/PublicationDetail?pub=pentenrieder2006giCirulis, A., & Brigmanis, K. B. (2013). 3D Outdoor Augmented Reality for Architecture and Urban Planning. Procedia Computer Science, 25, 71-79. doi:10.1016/j.procs.2013.11.009You, S., Neumann, U., & Azuma, R. (1999). Orientation tracking for outdoor augmented reality registration. IEEE Computer Graphics and Applications, 19(6), 36-42. doi:10.1109/38.799738Wither, J., Tsai, Y.-T., & Azuma, R. (2011). Indirect augmented reality. Computers & Graphics, 35(4), 810-822. doi:10.1016/j.cag.2011.04.010Radkowski, R., & Oliver, J. (2013). Natural Feature Tracking Augmented Reality for On-Site Assembly Assistance Systems. Lecture Notes in Computer Science, 281-290. doi:10.1007/978-3-642-39420-1_30Rao, J., Qiao, Y., Ren, F., Wang, J., & Du, Q. (2017). A Mobile Outdoor Augmented Reality Method Combining Deep Learning Object Detection and Spatial Relationships for Geovisualization. Sensors, 17(9), 1951. doi:10.3390/s17091951Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., & Stuetzle, W. (1993). Mesh optimization. Proceedings of the 20th annual conference on Computer graphics and interactive techniques - SIGGRAPH ’93. doi:10.1145/166117.166119Rossignac, J., & Borrel, P. (1993). Multi-resolution 3D approximations for rendering complex scenes. Modeling in Computer Graphics, 455-465. doi:10.1007/978-3-642-78114-8_29Gross, M. H., Staadt, O. G., & Gatti, R. (1996). Efficient triangular surface approximations using wavelets and quadtree data structures. IEEE Transactions on Visualization and Computer Graphics, 2(2), 130-143. doi:10.1109/2945.506225Botsch, M., Pauly, M., Rossl, C., Bischoff, S., & Kobbelt, L. (2006). Geometric modeling based on triangle meshes. ACM SIGGRAPH 2006 Courses on - SIGGRAPH ’06. doi:10.1145/1185657.1185839Pietroni, N., Tarini, M., & Cignoni, P. (2010). Almost Isometric Mesh Parameterization through Abstract Domains. IEEE Transactions on Visualization and Computer Graphics, 16(4), 621-635. doi:10.1109/tvcg.2009.96Khan, D., Yan, D.-M., Ding, F., Zhuang, Y., & Zhang, X. (2018). Surface remeshing with robust user-guided segmentation. Computational Visual Media, 4(2), 113-122. doi:10.1007/s41095-018-0107-yGuidi, G., Russo, M., Ercoli, S., Remondino, F., Rizzi, A., & Menna, F. (2009). A Multi-Resolution Methodology for the 3D Modeling of Large and Complex Archeological Areas. International Journal of Architectural Computing, 7(1), 39-55. doi:10.1260/147807709788549439Remondino, F., & El-Hakim, S. (2006). Image-based 3D Modelling: A Review. The Photogrammetric Record, 21(115), 269-291. doi:10.1111/j.1477-9730.2006.00383.xBruno, F., Bruno, S., De Sensi, G., Luchi, M.-L., Mancuso, S., & Muzzupappa, M. (2010). From 3D reconstruction to virtual reality: A complete methodology for digital archaeological exhibition. Journal of Cultural Heritage, 11(1), 42-49. doi:10.1016/j.culher.2009.02.006Unity, The Photogrammetry Workflowhttps://unity.com/solutions/photogrammetry.Blanco, S., Carrión, B., & Lerma, J. L. (2016). REVIEW OF AUGMENTED REALITY AND VIRTUAL REALITY TECHNIQUES IN ROCK ART. Proceedings of the ARQUEOLÓGICA 2.0 8th International Congress on Archaeology, Computer Graphics, Cultural Heritage and Innovation. doi:10.4995/arqueologica8.2016.3561Behzadan, A. H., & Kamat, V. R. (2010). Scalable Algorithm for Resolving Incorrect Occlusion in Dynamic Augmented Reality Engineering Environments. Computer-Aided Civil and Infrastructure Engineering, 25(1), 3-19. doi:10.1111/j.1467-8667.2009.00601.xTian, Y., Long, Y., Xia, D., Yao, H., & Zhang, J. (2015). Handling occlusions in augmented reality based on 3D reconstruction method. Neurocomputing, 156, 96-104. doi:10.1016/j.neucom.2014.12.081Tian, Y., Guan, T., & Wang, C. (2010). Real-Time Occlusion Handling in Augmented Reality Based on an Object Tracking Approach. Sensors, 10(4), 2885-2900. doi:10.3390/s10040288

Germline Mutations in FAN1 Cause Hereditary Colorectal Cancer by Impairing DNA Repair

Identification of genes associated with hereditary cancers facilitates management of patients with family histories of cancer. We performed exome sequencing of DNA from 3 individuals from a family with colorectal cancer who met the Amsterdam criteria for risk of hereditary nonpolyposis colorectal cancer. These individuals had mismatch repair-proficient tumors and each carried nonsense variant in the FANCD2/FANCI-associated nuclease 1 gene (FAN1), which encodes a nuclease involved in DNA inter-strand cross-link repair. We sequenced FAN1 in 176 additional families with histories of colorectal cancer and performed in vitro functional analyses of the mutant forms of FAN1 identified. We detected FAN1 mutations in approximately 3% of families who met the Amsterdam criteria and had mismatch repair-proficient cancers with no previously associated mutations. These findings link colorectal cancer predisposition to the Fanconi anemia DNA repair pathway, supporting the connection between genome integrity and cancer risk

Revisiting the usefulness of the short acute octreotide test to predict treatment outcomes in acromegaly

Introduction: We previously described that a short version of the acute octreotide test (sAOT) can predict the response to first-generation somatostatin receptor ligands (SRLs) in patients with acromegaly. We have prospectively reassessed the sAOT in patients from the ACROFAST study using current ultra-sensitive GH assays. We also studied the correlation of sAOT with tumor expression of E-cadherin and somatostatin receptor 2 (SSTR2) .Methods: A total of 47 patients treated with SRLs for 6 months were evaluated with the sAOT at diagnosis and correlated with SRLs' response. Those patients whose IGF1 decreased to = 3SDS, were considered non-responders. The 2 hours GH value (GH2h) after s.c. administration of 100 mcg of octreotide was used to define predictive cutoffs. E-cadherin and SSTR2 immunostaining in somatotropinoma tissue were investigated in 24/47 and 18/47 patients, respectively.Results: In all, 30 patients were responders and 17 were non-responders. GH(2h) was 0.68 (0.25-1.98) ng/mL in responders vs 2.35 (1.59-9.37) ng/mL in non-responders (p<0.001). GH(2h) = 1.4ng/mL showed the highest ability to identify responders (accuracy of 81%, sensitivity of 73.3%, and specificity of 94.1%). GH(2h) = 4.3ng/mL was the best cutoff for non-response prediction (accuracy of 74%, sensitivity of 35.3%, and specificity of 96.7%). Patients with E-cadherin-positive tumors showed a lower GH(2h) than those with E-cadherin-negative tumors [0.9 (0.3-2.1) vs 3.3 (1.5-12.1) ng/mL; p<0.01], and patients with positive E-cadherin presented a higher score of SSTR2 (7.5 +/- 4.2 vs 3.3 +/- 2.1; p=0.01).Conclusion: The sAOT is a good predictor tool for assessing response to SRLs and correlates with tumor E-cadherin and SSTR2 expression. Thus, it can be useful in clinical practice for therapeutic decision-making in patients with acromegaly

A Novel Monoclonal Antibody Targeting a Large Surface of the Receptor Binding Motif Shows Pan-neutralizing SARS-CoV-2 Activity Including BQ.1.1 Variant

In the present study we report the functional and structural characterization of 17T2, a new highly potent pan-neutralizing SARS-CoV-2 human monoclonal antibody (mAb) isolated from a convalescent COVID-19 individual infected during the first wave of the COVID-19 pandemic. 17T2 is a class 1 VH1-58/κ3-20 antibody, derived from a receptor binding domain (RBD)-specific IgA memory B cell and developed as a human recombinant IgG1. Functional characterization revealed that 17T2 mAb has a high and exceptionally broad neutralizing activity against all SARS-CoV-2 spike variants tested, including BQ.1.1. Moreover, 17T2 mAb has in vivo prophylactic activity against Omicron BA.1.1 infection in K18-hACE2 transgenic mice. 3D reconstruction from cryogenic-electron microscopy (cryo-EM) showed that 17T2 binds the Omicron BA.1 spike protein with the RBD domains in up position and recognizes an epitope overlapping with the receptor binding motif, as it is the case for other structurally similar neutralizing mAbs, including S2E12. Yet, unlike S2E12, 17T2 retains its high neutralizing activity against all Omicron sublineages tested, probably due to a larger contact area with the RBD, which could confer a higher resilience to spike mutations. These results highlight the impact of small structural antibody changes on neutralizing performance and identify 17T2 mAb as a potential candidate for future therapeutic and prophylactic interventions.We acknowledge access to the cryo-EM CNB-CSIC facility in the context of the CRIOMECORR project (ESFRI-2019-01-CSIC-16) and we thank the staff of the Protein Technology Unity (CRG) for the help in protein production. This study was supported by the COVID-19 call grant from Generalitat de Catalunya, Department of Health (to GM), grant Miguel Servet research program (to GM), and partially funded by the crowdfunding initiative #joemcorono and the Fundació Glòria Soler (to JB). A.P-G. was supported by a predoctoral grant from Generalitat de Catalunya and Fons Social Europeu (2022 FI_B 00698).N